1. Field of the Invention
The present invention generally relates to a learning type recognition & judgement apparatus performing a learning operation of an input pattern data and recognizing the same.
2. Prior Art
According to this kind of conventional apparatus, an input pattern data is roughly classified to select the category group to which the input data belongs. Next, in the selected category group, fine classification is performed to recognize the input data. For example, this kind of learning type recognition & judgement apparatus is disclosed in "A Large Scale Neutral Network CombNET-II", Paper Journal of the Institute of Electronic Information and Communications, D-II Vol. J75-D-II No.3 pp545-553.
FIG. 15 is a schematic block diagram showing an arrangement of a conventional learning type recognition & judgement apparatus. In FIG. 15, reference numeral 1 represents a rough classification section which comprises a plurality of input sections 1a and a plurality of multi input/output signal processing sections 1b. With this arrangement, rough classification section 1 calculates an output value (adaptation or fidelity) of each category group with respect to the input pattern signal X, thereby roughly classifying the input pattern signal into respective category groups. Reference numeral 2 represents each of a plurality of fine classification sections. Each fine classification section 2 comprises a plurality of input section 2a, a plurality of multi input/output signal processing sections 2b and a maximum value selecting section 2c. With this arrangement, each fine classification section 2 finely classifies the input signal in each category group.
More specifically, a fine classification input signal selecting section 4 generates the input pattern signal which is entered into each fine classification section 2 through input sections 2a. Each multi input/output signal processing section 2b multiplies the output of an associated lower-layer input section 2a or multi input/output signal processing section 2b with a weighting coefficient reflecting a degree of connection, and sums up resultant multiplication results. Then, thus obtained summation value is processed by an appropriate threshold, and is then outputted.
These plural multi input/output signal processing section 2b are arranged in a multi-layer structure without no mutual connection in each layer. And the network is constituted so as to propagate signals only to upper layers, thereby obtaining the similarity degree of the input pattern signal to each category in the category group. Maximum value selecting section 2c selects the maximum value among the outputs generated from plural multi input/output signal processing sections 2b of the most-highest layer.
Reference numeral 3 represents a group selecting section which selects a plurality of category groups among the output values (adaptations) of rough classification section 1. The above-described fine classification section input signal selecting section 4 selects a fine classification section 2, to which the input pattern signal is entered, based on the group selecting information obtained in group selecting section 3. Reference numeral 5 represents a discriminating section which comprises a plurality of similarity calculating sections 5a and a category discriminating section 5b. Each similarity calculating section 5a calculates a similarity of each category based on the adaptation of the category group selected by group selecting section 3 and the output value of the fine classification section 2 corresponding to its category group. Then, category discriminating section 5b finds out the maximum value of the resultant similarities of categories, thereby discriminating the input pattern signal X.
An operation of the above-described conventional learning type recognition & judgement apparatus will be explained, hereinafter. Input pattern signal X, consisting of n feature data of an object to be recognized, is expressed by the following equation 1. EQU X=(x.sub.1,x.sub.2, - - - ,x.sub.n) (1)
This input pattern signal X is first entered into the input sections 1a of rough classification section 1. There are provided a total of "n" input sections 1a; this number "n" is; equal to the number of feature data of the pattern data. Each feature data x.sub.i is entered into a corresponding input section 1a. Each multi input/output signal processing section 1b multiplies each input x.sub.j entered from its associated input section 1a with a relevant weighting coefficient v.sub.ij (1.ltoreq.i.ltoreq.m.sub.r ; m.sub.r is the number of category group, 1.ltoreq.j.ltoreq.n) representing the degree of linkage therebetween, and obtains the sum of all multiplication results. As expressed by the following equation: 2, the obtained result is referred to as a weighting coefficient vector V.sub.i of the input pattern signal X and each multi. input/output signal processing section 1b. EQU V.sub.i =(v.sub.i1,v.sub.i2, - - - ,v.sub.in) (2)
Multi input/output signal processing section 1b outputs a value obtained by dividing the multiplication result by the norm .vertline.X.vertline., .vertline.V.sub.i .vertline. of the equations (1) and (2). More specifically, the output value sim (X,V.sub.i) of a multi input/output signal processing section 1b is expressed by the following equation 3 using its weighting coefficient vector V.sub.i. EQU sim (X,V.sub.i)=(X.multidot.V.sub.i)/(.vertline.X.vertline..multidot..vertline .V.sub.i .vertline.) (3)
where X.multidot.V.sub.i =.SIGMA..sub.j (x.sub.j .multidot.v.sub.ij)
.vertline.X.vertline.=(.SIGMA.x.sub.j.sup.2).sup.0.5 PA2 .vertline.Vi.vertline.=(.SIGMA.v.sub.ij.sup.2).sup.0.5
Weighting coefficient vector V.sub.i is designed beforehand too let a predetermined multi input/output signal processing section 1b generate a maximum output in response to a similar input pattern signal.
This kind of weighting coefficient vector Vi has been conventionally designed in the following manner. First of all, in a first step, in response to each entry of the input pattern signal X for designing the weighting coefficient vector, a vector V.sub.c having the most-largest sim (X, V.sub.i) is obtained to bring V.sub.c into a closer relation to X. In other words, X is optimally matched with V.sub.c. If a predetermined number of input pattern signals are optimally matched with a particular weighting coefficient vector, a region covered by this weighting coefficient vector is divided into two.
Next, in a second step, optimal matching vector V.sub.i is obtained for each of all the input pattern signals used for designing the weighting coefficient vector. Then, it is checked whether there is any change in a comparison between a resultant one and the previous one. If any change is found, a corresponding vector V.sub.i is modified. In this case, the region covered by this weighting coefficient vector is divided appropriately in the same manner as in the first step. The second step is repetitively performed until modification and division are no longer required.
By executing the design of the weighting coefficient vector in this manner, the input pattern signal is roughly classified into a plurality of category groups. The output of each multi input/output signal processing section 1b, which represents an adaptation of each category group with respect to the input pattern signal X, is sent to group selecting section 3.
Group selecting section 3 selects an arbitrary number of category groups in order of the largeness of the adaptation obtained in rough classification section 1, and separately generates the group selecting information indicating the selected category groups and the corresponding adaptations. Based on the group selecting information obtained from group selecting section 3, fine classification input signal selecting section 4 selects fine classification sections 2 into which the input pattern signal should be entered, and sends the input pattern signal to these selected fine classification sections 2.
In each of the fine classification sections 2 corresponding to the category groups selected by group selecting section 3 (i.e. fine classification sections 2 which received the input pattern signal X from fine classification input signal selecting section 4), the input pattern signal X is first entered into input sections 2a. There are provided a total of "n" input sections 2a; this number "n" is equal to the-number of feature data of the input pattern signal X. Each feature data x.sub.i is entered into its corresponding input section 2a. Each multi input/output signal processing section 2b in the fine classification section 2 multiplies the output of an associated lower-layer input section 2a or multi input/output signal processing section 2b with a weighting coefficient representing the degree of connection, and sums up resultant multiplication results. Then, thus obtained summation value is converted through an appropriate threshold function, and is then sent to its upper layer. The total number of multi input/output signal processing section 2b constituting the most-highest layer is identical with the number of categories of the pattern data involved in each category group. Hence, each multi input/output signal processing section 2b of the most-highest layer is related to each of these categories in a one-to-one relationship. Maximum value selecting section 2c selects the maximum one of outputs generated from multi input/output signal processing sections 2b of the most-highest layer, and sends out this maximum output value together with a category corresponding to the selected multi input/output signal processing section 2b.
The weight coefficients of each multi input/output signal processing section 2b are learned beforehand in such a manner that, when the input pattern signal involves each category in the category group, the highest-layer multi input/output signal processing section 2b corresponding to each category always generates the maximum output.
More specifically, this kind of weighting coefficient learning operation is carried out by the learning algorithm generally called the "back-propagating error method", which is disclosed, for example, in "Learning Representations by Back-Propagating Errors," by D. E. Rumelhart, G. E. Hinton and R. J. Williams, Nature, vol.323, pp.533-536, Oct. 9, 1986.
Hereinafter, an abstract of the back-propagating error method will be explained.
First, the input pattern signal X used for learning the weighting coefficients is entered into input sections 2a of fine classification section 2. As described above, each multi input/output signal processing section 2b in the fine classification section 2 multiplies the output of an associated lower-layer input section 2a or multi input/output signal processing section 2b with a weighting coefficient representing the degree of connection, and sums up resultant multiplication results. Thus obtained summation value is converted through an appropriate threshold function, and is then sent to its upper layer.
Assuming that "o.sub.k " represents an output signal of all of multi input/output signal processing sections 2b of the most-highest layer while "t.sub.k " represents a desirable output signal (which is generally called "teacher signal"), an error "E" is defined by the following equation 4. EQU E=0.5.SIGMA..sub.p .SIGMA..sub.k (t.sub.k -o.sub.k).sup.2 (4)
where .SIGMA..sub.p represents the sum relating to the pattern number of the teacher signal. The purpose of the learning operation is to determine a weighting coefficient which minimizes this error E. A change amount .DELTA.w.sub.ij of the weighting coefficient between multi input/output signal processing sections 2b is calculated based on the following equation 5. EQU .DELTA.w.sub.ij =-.epsilon..delta.E/.delta.w.sub.ij (5)
where .epsilon. represents a positive constant which is generally called the learning rate.
Error E can be reduced by repeating the renewal of the weighting coefficient based on the equation 5 in response to each entry of the learning pattern signal. When the error E becomes a small value enough to judge that the output signal is almost equalized with the desirable value, the learning operation is terminated.
By using such a weighting coefficient learning method, it becomes possible to set optimal weight coefficients capable of letting the highest-layer multi input/output signal processing section 2b corresponding to each category always generates the maximum output when the input pattern signal involves each category in the category group. Accordingly, it becomes possible to recognize the category of the input pattern signal in each category group, i.e. in each fine classification section, in accordance with the selecting operation of the maximum value selecting section 2c which identifies a specific highest-layer multi input/output signal processing section 2b generating the maximum output.
In the discriminating section 5, each similarity calculating sections 5a calculates a similarity of each category obtained in the fine classification section 2 based on the adaptation of the category group selected by group selecting section 3 and the output value of the fine classification section 2 corresponding to its category group, using the following equation 6. EQU (Similarity)=(Adaptation).sup.a .multidot.(Output value).sup.b(6)
where a and b are real constants.
Finally, category discriminating section 5b compares the resultant similarities of respective categories obtained from similarity calculating sections 5a, and finds out the category having the maximum similarity, thereby outputting a final discriminating result.
However, according to the conventional above-described arrangement, the learning operation in each fine classification section 2 is independently performed without considering the distance, i.e. group attribution factor, between the group reference pattern and the input pattern signal to be learned. Hence, there was a problem that the recognition accuracy would be undesirably deteriorated when the input pattern signal was positioned on the boundary of category groups. Furthermore, if a non-learned learning pattern is erroneously recognized or rejected nevertheless it's correctness, it is no longer impossible to cure or undo it.
Moreover, when the recognition is rejected, it was impossible to perform a reject processing with reference to the category groups or categories of the rejected data.